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Abstract — We present and analyze a joint source-channel 
coding strategy for the transmission of a Gaussian source across 
a Gaussian channel in n channel uses per source symbol. Among 
all such strategies, the scheme presented here has the following 
properties: i) the resulting mean-squared error scales optimally 
with the signal-to-noise ratio, and ii) the scheme is easy to 
implement and the incurred delay is minimal, in the sense that 
a single source symbol is encoded at a time. 

1. Introduction 

In this paper we propose and analyze a scheme for the 
transmission of a discrete-time memoryless Gaussian source 
across a discrete-time memoryless Gaussian channel, where 
the channel can be used n times for each source symbol. The 
parameter n is arbitrary but fixed, given as part of the problem 
statement. 

It is well known that if the source has variance cr| and 
the channel noise has variance (t| then the average transmit 
power P and the average mean-squared error D of any 
communication scheme for this scenario are related by 



X2 



R{D) < nC{P), 



(1) 



where R{D) = 0.5 log(cr|/Z3) is the rate-distortion function 
of the source and C{P) — 0.5 log(l + -P/cr^) is the capacity- 
cost function of the channel (see e.g. H]). Inserting into ^ 
yields 



D 



< 1 



P 



or equivalently 



SDR < (1 + SNR)", 



where we have defined SNR 
the limit when SNR goes to infinity, 

log SDR 



and SDR 



lim 

SNR^oo log SNR 



< n. 



(2) 

allD. In 
(3) 



At large SNR, the SDR (signal-to-distortion ratio) behaves 
thus at best as SNR". In this sense n is the best possible scaling 
exponent that any communication scheme can hope to achieve 
for a fixed n. 

The scheme proposed in this paper achieves this optimal 
scaling exponent for any fixed n, yet has small complexity and 
minimal delay in the sense that it operates on a single source 
symbol at a time. It works by quantizing the source and then 
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(a) Shannon's original proposition. 



(b) Mapping proposed in tliis paper 
(for n = 2). 



Fig. 1. A minimum-delay source-channel code for n = 2 can be visualized 
as a curve in parametrized by the source. Here we compare the mapping 
presented in this paper (right) to Shannon's oiiginal suggestion (left). 



repeatedly quantizing the quantization error. The quantized 
points are sent across the first n — 1 channel uses and the 
last quantization error is sent uncoded in the n*^ channel use. 

If the quantization resolution is chosen correctly (as a func- 
tion of the SNR), then the decoding error of the quantization 
symbols is dominated by that of the uncoded transmission in 
the last channel use, which is shown to have an asymptotic 
scaling exponent of n. 

Schemes similar to the one proposed here have been con- 
sidered before. Indeed, one of the first schemes to transmit 
an analog source across two uses of a Gaussian channel 
was suggested by Shannon |2|. Notice its resemblance to the 
constellation studied here, shown in Figure [T] 

After Shannon, Wozencraft and Jacobs ||3l were among 
the first to study source-channel mappings as curves in n- 
dimensional space. Ziv [|4j| found important theoretical limi- 
tations of such mappings. Much of the later work is due to 
Ramstad and his coauthors (see 13], 0, Q, lEl, |j9], flOl). A 
proof that the performance of minimal-delay codes is strictly 
smaller than that of codes with unrestricted delay when n > 1 
was given in 2008 by Ingber et al. fTTl . 

For n — 1, the presented scheme is almost identical to the 
HSQLC scheme by Coward fT2], which uses a numerically 
optimized quantizer, transmitter and receiver to minimize the 
mean-squared error (MSE) for finite values of the SNR. 
Coward correctly conjectured that the right strategy for n > 2 



would be to repeatedly quantize the quantization error from 
the previous step, which is exactly what we do here. 

Another closely related communication scheme is the shift- 
map scheme due to Chen and Wornell llT3l . Vaishampayan and 
Costa |T4] showed in their analysis that it achieves the scaling 
exponent n — e for any e > if the relevant parameters are 
chosen correctly as a function of the SNR. Up to rotation and 
a different constellation shaping, the shift-map scheme is in 
fact virtually identical to the one presented here, a fact that 
was pointed out recently by Taherzadeh and Khandani IfTSl . 
In their own paper they develop a scheme that achieves the 
optimal scaling exponent exactly and is in addition robust to 
SNR estimation errors; their scheme, however, is based on 
rearranging the digits of the binary expansion of the source 
and is thus quite different from the one presented here. 

Shamai, Verdii and Zamir lfT6l used Wyner-Ziv coding to 
extend an existing analog system with a digital code when 
additional bandwidth is available. Mittal and Phamdo 1, 17J (see 
also the paper by Skoglund, Phamdo and Alajaji ifTSl ) split 
up the source into a quantized part and a quantization error, 
much like we do here, but they use a separation-based code (or 
"tandem" code) to transmit the quantization symbols. Reznic 
et al. 1191 use both quantization and Wyner-Ziv coding, and 
their scheme includes Shamai et al. and Mittal & Phamdo as 
extreme cases. AU three schemes, however, use long block 
codes for the digital phase and incur correspondingly large 
delays, so they are not directly comparable with minimum 
delay schemes. 

While the basic idea of the scheme considered in this paper 
is not new, the analysis provided is and, to our knowledge, we 
are the first to give an exact mathematical formulation of the 
quantization resolution (as a function of the SNR) that leads 
to the optimal scaling exponent. 

II. Proposed Communication Scheme 
A. Encoder 

To encode a single source letter S into n channel input 
symbols Xi, . . . , X„, we proceed as follows. Define Eq — S 
and recursively compute the pairs (Qi, Ei) as 

g, = 4 int 1) 



E, = 1 - Q,) (4) 

71—1 where int(a::) is the unique integer i such 



for i — 1, 
that 



1 



1 



2' 2 



and /3 is a scaling factor that grows with the power P in a way 
to be determined later. The following result will be useful in 
the sequel. 

Lemma 1: As (3 goes to infinity, the variance of Q,: con- 
verges to that of Ei^i for all z = 1, . . . , n — 1. 

Proof: Intuitively this is so since Qi is quantized, 
and the quantization step becomes smaller as (3 goes to infinity. 
A rigorous proof is given in Appendix lAl ■ 



Proposition 2: The Qi and Ei satisfy the following prop- 
erties: 

1) The map S i— > (Qi, . . . , Qn-i, En^i) is one-to-one and 



1 



1 



(5) 



2) The variance of Eq is cr|. and for all i = 1, . . . , n — 1, 

Ei e [-1/2, 1/2) and Var(£;,) < 1/4. 

3) For any ^ > there exists /?o such that for (3 > Pa, 



\ <Jq + S for i = 1 



1/4 + 6 for i = 2, n - 1. 



(6) 



Proof: 

1) From the definition (|4| we have 

1 

Ei-i ~ —Ei -i 
P 



(7) 



Repeated use of this relationship leads to the given 
expression for S. 

2) First, Var(£:o) = Var(S') = ct|. Next, E^ e [-1/2, 1/2) 
follows trivially from the definition of Ei. Furthermore, 
the variance of any random variable with support in an 
interval of length 1 is bounded from above by 1/4. 

3) The result follows directly from Lemma [1] and from the 
bound on the variance of the Ei in point 2 above. 

■ 

Without loss of generality we assume hereafter that ct| > 
1/4 so that the first bound of (|6]l applies to all Qi. 

We determine the channel input symbols Xi from the Qi 
and from £'„_i according to 



X, = 



Xn — 




for i — 1, 



71—1 and 



where ct|; = Var(i?,i_i). Following Proposition|2] this ensures 
that ^[Xf] < P for all i and for /3 > f3o{d). Since we are 
interested in the large SNR regime and since we have defined 
(3 to grow with P, we can thus assume for the remainder that 
the power constraint is satisfied. 

B. Decoder 

The Xi are transmitted across the channel, producing at the 
channel output the symbols 



. ,71, 



where the Zi are iid Gaussian random variables of vari- 
ance (T^. To estimate S from Yi, F„, the decoder first 
computes separate estimates Qi, Qn-i and En-i, and 
then combines them to obtain the final estimate S. While this 
strategy is suboptimal in terms of achieving a small MSB, we 
will see that it is good enough to achieve optimal scaling. 



To estimate the Qt we use a maximum likelihood (ML) 
decoder, which yields the minimum distance estimate 



1 

- arg mm 

13 jez 



P J 

1 + 513 



-Y,, 



(8) 



To estimate i?„-i, we use a linear minimum mean-square 
error (LMMSE) estimator (see e.g. |,20i Section 8.3]), which 
computes 



En-1 



-Y„, 



Finally we use the relationship (|5]l to obtain 



5" = 



n— 1 ^ 



-E„ 



(9) 



(10) 



C. Error Analysis 

The overall MSB E[{S - Sf] can be broken up into 
contributions due to the errors in decoding Qi and En-i as 
follows. From (|5]l and (fTOl i. the difference between S and S 
is 

n-l 

The error terms Qi — Qi depend only on the noise of the 
respective channel uses and are therefore independent of each 
other and of £'„_i — En-i, so we can write the error variance 
componentwise as 

n-l 

ms - sr] = E ^^sq, + ^^^se, (11) 



^2(n-l)' 



where fg,, E[(Q, - Qif] and £e = HiEn-i - E„-if] 
Lemma 3: For each i ~ 1, . . . , n — 1, 



dof - 



£Q.^ e O (exp{-fcSNR//32}) , 

where SNR = P/t^z fc > is a constant. 
(The 0-notation is defined in Appendix iBl) 
Proof: Define the interval 



(12) 



According to the minimum distance decoder (O, Qi — Qi — 
j /f3 whenever Zi E Tj. The error Eq^i satisfies thus 



f3' 



(13) 



3 = i 



where the second equaUty follows from the symmetry of the 
distribution of Zi. Now, 



Pr[z, e Jj] = Q ( ;7 ^ I - ' ^ 



'SNR 



JT5 



where 



Q{x) 



which can be bounded from above for a; > as 

Q(x) < ie-V2. 
For > 1 we can now bound ( fT3] l as 

(j - 1/2)2SNR 



Note that for j > 2, (j - 1/2)2 > ^ -pj^^^ 



^^Q,i < exp 



SNR 



E 

i=2 



j2 exp 



:;SNR 



2/32(a| + J) 



To bound the infinite sum we use 



p2 +p 



i=2 



(14) 



(15) 



with p = exp{-SNR/2/32(cr| + 5)}. The first term of (flU) 
thus dominates for large values of SNR//?^ and 



'S'q,! < cexp 



SNR 



for some c > 0, which completes the proof. ■ 
Lemma 4: Ee G 0(snr^^). 

Proof: The mean-squared error that results from the 
LMMSE estimation ^ is 



Since 



_ 2 (EK_iy„])2 
" ~ E[y„2] • 



y„ — X„ + Zn = \ -^En-l + Z„ 



(16) 



we have E[£;„_iF„] = Moreover, E[i;,2] = E[X^ 

E[Z^] = P + a^. Inserting this into we obtain 



Ee = cte ^ 



P-l 
P + al 



al ( 1 



(7 



P 



P + al 



E 



< 



1 + SNR 
^1 



(72 



SNR 



Since (t|; is bounded (cf. Proposition |2]i, Ee G 0(snr ^) as 
claimed. ■ 



D. Achieving the Optimal Scaling Exponent 
Recall the formula for the overall error 

""^ 1 1 

E[(S' - Sf] = i^2{^-l)^Q-^ + lj2(n-l)^E. 

According to Lemma |3] Eq ^ decreases exponentially when 
SNR//3^ goes to infinity. This happens for increasing SNR if 
we set e.g. 

for some e > 0, in which case £q ^ e O (exp(— feSNR*^)). 
From this and Lemma |4] the overall error satisfies 



(17) 



where e' = (n — l)e can be made as small as desired. The 
scaling exponent for a fixed e satisfies therefore 

log SDR logcrl + (n - e') logSNR , 
lim > lim ^ — ; = n—e . 



log SNR 



SNR— KDO 



log SNR 



(18) 



Note that the choice of e represents a tradeoff: for small e 
the error due to the "discrete" part vanishes only slowly, but 
the scaling exponent in the limit is larger. For larger e, £q 
vanishes quickly but the resulting exponent is smaller. In the 
remainder of this section we show how we can choose e as a 
function of SNR to achieve the optimal scaling. 

Let now 



e(SNR) 



log(?llogSNR/fc) 



(19) 



log SNR 

where k is the constant indicating the decay of £q ^ in ( fT2] i. 
With this choice of e, 

Sg^i e O (exp(-fcSNR')) 
= 0(SNR-"), 

hence the overall error is still dominated as in ([TtI i. and ( fTSl l 
still applies. Inserting ST9[ in (fTsT l. we find 



lim 

SNR^oo 



logcr| + (n - (n - l)e) log SNR 



= lim 

SNR^OO 



log SNR 

logcr| + n log SNR — (n — 1) log(nlog SNR/fc) 

log SNR 



which is indeed the optimal scaling exponent. 

Remark 1: While the limiting exponent above is indeed the 
optimal one, the SDR scales as SNR"(log SNR)^("^^) rather 
than the theoretic optimum SNR". This means that the gap 
(in dB) between the theoretically optimal SDR value and our 
lower bound grows to infinity as SNR oo. According to 
a result in |15|, however, no scheme combining quantization 
and uncoded transmission as done here can achieve a better 
SDR scaling than SNR"(log SNR)^^"^^). In the scaling sense, 
our bound is therefore tight. 



III. Conclusions 

We have presented and analyzed a joint source-channel 
communication strategy that achieves the optimal scaling 
exponent if the channel is to be used n times per source 
symbol. The given scheme incurs the smallest possible delay 
and its implementation is straightforward. 

While the basic structure of this scheme - separating the 
source into a quantized part and the associated error - is 
not new, the simple analysis provided here yields an explicit 
expression for the quantization resolution in terms of the SNR 
that leads to the optimal scaling exponent. 

Appendix A 
Proof of Lemma[T] 

Since all involved distributions are symmetric, K[Qi] ~ 0. 
Writing Qi as a function of we have 

/oo 
Q^{0^f{Odt (20) 
-oo 

where /(^) is the pdfl of Ei-i. Now, Q^(<f) = j /f3 whenever 
With this, the integral ( l20b becomes 



Var(Q,) 



3 + 1/2 



E 



/2 



where F{S^) is the cd^ of Ei^i. As /? goes to infinity, this 
sum converges to a Riemann-Stieltjes integral: 



Var(g,) ^ j edF{0 = Var(£;,_i) 



as /3 - 



Appendix B 
BiG-O Notation 

The "Big-O" asymptotic notation used at various points in 
the paper is defined as follows. Let f{x) and g{x) be two 
functions defined on R. We write 

f(x) e Oiaix)) 

if and only if there exists an xq and a constant c such that 

fix) < cg{x) 

for all X > xq. 

As a simple consequence of this definition, if f{x) G 0(x") 
then 

log/(x) 
lim — < n. 



logx 



'probability density function 
^cumulative distribution function 
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